Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Survey of communication overhead of federated learning
Xinyuan QIU, Zecong YE, Xiaolong CUI, Zhiqiang GAO
Journal of Computer Applications    2022, 42 (2): 333-342.   DOI: 10.11772/j.issn.1001-9081.2021020232
Abstract1822)   HTML290)    PDF (1356KB)(2345)       Save

To solve the irreconcilable contradiction between data sharing demands and requirements of privacy protection, federated learning was proposed. As a distributed machine learning, federated learning has a large number of model parameters needed to be exchanged between the participants and the central server, resulting in higher communication overhead. At the same time, federated learning is increasingly deployed on mobile devices with limited communication bandwidth and limited power, and the limited network bandwidth and the sharply raising client amount will make the communication bottleneck worse. For the communication bottleneck problem of federated learning, the basic workflow of federated learning was analyzed at first, and then from the perspective of methodology, three mainstream types of methods based on frequency reduction of model updating, model compression and client selection respectively as well as special methods such as model partition were introduced, and a deep comparative analysis of specific optimization schemes was carried out. Finally, the development trends of federated learning communication overhead technology research were summarized and prospected.

Table and Figures | Reference | Related Articles | Metrics
Robust speech recognition technology based on self-supervised knowledge transfer
Caitong BAI, Xiaolong CUI, Huiji ZHENG, Ai LI
Journal of Computer Applications    2022, 42 (10): 3217-3223.   DOI: 10.11772/j.issn.1001-9081.2021050808
Abstract241)   HTML7)    PDF (2421KB)(58)       Save

A robust speech recognition model training algorithm based on self-supervised knowledge transfer was proposed to solve the problems of the increasingly high cost of tagging neural network training data and noise interference hindering performance improvement of speech recognition system. Firstly, three artificial features of the original speech samples were extracted in the pre-processing stage. Then, the advanced features generated by the feature extraction network were fitted to the artificial features extracted in the pre-processing stage through three shallow networks respectively in the training stage. At the same time, the feature extraction front-end and the speech recognition back-end were cross-trained, and their loss functions were integrated. Finally, the advanced features that are more conducive to denoised speech recognition were extracted by the feature extraction network after using the gradient back propagation, thereby realizing the artificial knowledge transfer and denoising as well as using training data efficiently. In the application scenario of military equipment control, the word error rate of the proposed method can be reduced to 0.12 based on the test on three open source Chinese speech recognition datasets THCHS-30 (TsingHua Continuous Chinese Speech), Aishell-1 and ST-CMDS (Surfing Technology Commands) as well as the military equipment control command dataset. Experimental results show that the proposed method can not only train robust speech recognition models, but also improve the utilization rate of training samples through self-supervised knowledge transfer, and can complete equipment control tasks.

Table and Figures | Reference | Related Articles | Metrics